Proportional Volume Sampling and Approximation Algorithms for A-Optimal Design

We study the A-optimal design problem where we are given vectors v1, . . . , vn ∈ R, an integer k ≥ d, and the goal is to select a set S of k vectors that minimizes the trace of (∑ i∈S viv > i )−1 . Traditionally, the problem is an instance of optimal design of experiments in statistics (Pukelsheim (2006)) where each vector corresponds to a linear measurement of an unknown vector and the goal is to pick k of them that minimize the average variance of the error in the maximum likelihood estimate of the vector being measured. The problem also finds applications in sensor placement in wireless networks (Joshi and Boyd (2009)), sparse least squares regression (Boutsidis et al. (2011)), feature selection for k-means clustering (Boutsidis and Magdon-Ismail (2013)), and matrix approximation (de Hoog and Mattheij (2007, 2011); Avron and Boutsidis (2013)). In this paper, we introduce proportional volume sampling to obtain improved approximation algorithms for A-optimal design. Given a matrix, proportional volume sampling involves picking a set of columns S of size k with probability proportional to μ(S) times det( ∑ i∈S viv > i ) for some measure μ. Our main result is to show the approximability of the A-optimal design problem can be reduced to approximate independence properties of the measure μ. We appeal to hard-core distributions as candidate distributions μ that allow us to obtain improved approximation algorithms for the Aoptimal design. Our results include a d-approximation when k = d, an (1 + )-approximation when k = Ω ( d + 1 2 log 1 ) and k k−d+1 -approximation when repetitions of vectors are allowed in the solution. We also consider generalization of the problem for k ≤ d and obtain a kapproximation. The last result also implies a restricted invertibility principle for the harmonic mean of singular values. We also show that the A-optimal design problem is NP-hard to approximate within a fixed constant when k = d.